Skip to content

feat: git sync — webhook, prune, manual flow (Plan C of issue #16)#135

Closed
michaelmcnees wants to merge 14 commits into
feat/git-sync-enginefrom
feat/git-sync-complete
Closed

feat: git sync — webhook, prune, manual flow (Plan C of issue #16)#135
michaelmcnees wants to merge 14 commits into
feat/git-sync-enginefrom
feat/git-sync-complete

Conversation

@michaelmcnees

Copy link
Copy Markdown
Collaborator

Summary

Plan C of 3 for issue #16. Stacked on #134 (Plan B sync engine), which is stacked on #133 (Plan A foundation). This PR finishes the core GitOps loop.

  • Prune — migration 020 adds workflow_definitions.disabled_at and a git_repo_workflows join table. The sync engine records each seen workflow per repo, then disables any that disappeared when prune: true. Reappearing files clear disabled_at on the next sync. History preserved.
  • Webhook receiverPOST /hooks/git/<repo-id> verifies HMAC-SHA256 via X-Hub-Signature-256, emits git.push.received, and fires SyncRepo in a goroutine so the provider gets its 202 fast. Body size capped at 1 MiB. Constant-time HMAC comparison.
  • Manual flowmantle repos plan <name> previews pending changes without writing; mantle repos apply <name> syncs on demand for auto_apply: false repos.
  • Delete cleanuprepo.Store.Delete now removes the cloned working tree under <artifact>/git/<repo-id>/ when a repo is unregistered. DB ON DELETE CASCADE handles the join-table rows automatically.
  • URL sanitizationsanitizeURL strips user:password@ from URLs in audit metadata and last_sync_error, so go-git errors echoing a clone URL can't leak inline credentials.
  • mantle repos update — wires the existing Store.Update into a CLI subcommand for in-place edits.
  • --webhook-secret flag on repos add and repos update so operators can arm HMAC verification without raw SQL.
  • Audit eventsgit.sync.pruned, git.push.received.

What operators can do with A + B + C

  • Register repos (CLI or git_sync.repos: in mantle.yaml)
  • Run mantle serve — reconciler materializes config, poller syncs each enabled auto_apply: true repo at poll_interval
  • Set up push webhooks (POST /hooks/git/<id> + X-Hub-Signature-256) for immediate syncs
  • Preview (plan), manually apply (apply), or force-sync (sync) on any repo
  • Delete a repo — DB row + cloned working tree + join-table rows all clean up
  • Pruned workflows survive as disabled versions in history, ready to be re-enabled if the file reappears

This closes the core GitOps loop that issue #16 targets. Deferred to Plan D (future work, non-blocking): Prometheus metrics, REST API for repo CRUD, SSH key credential auth, live poller reconfig (new repos picked up without restart), k8s git-sync sidecar deployment docs, webhook_secret encryption at rest, prune-skip observability surfacing.

Pre-push review fixes (commit 0ca3f60)

Four reviewer agents ran before push (Technical Writer, Product Manager, Legal Compliance, Reality Checker). Consolidated fixes:

  • Blocker: --webhook-secret flag added to repos add and repos update — operators can now arm HMAC verification through the CLI; the value is never printed to stdout (verified via test).
  • Removed stray fmt.Println("ok") from the webhook test.
  • Clarified the plan, apply, and update subcommand Long text; the parent repos Long now mentions all four sync-adjacent commands.
  • Extended sanitizeURL wrapping to the last_sync_error path (pull-failed / discover-failed / aggregated failure summary) so the column surfaced by mantle repos status never displays a credentialed URL.

Test plan

  • go test ./... in packages/engine — all packages green
  • go test ./internal/repo/sync/ — 20 tests including prune, URL sanitization, PlanRepo
  • go test ./internal/repo/ — 19 tests including GetByID, Delete cleanup, webhook_secret persistence
  • go test ./internal/server/ — 3 new webhook tests (202 / 403 / 404)
  • go test ./internal/cli/ — 10 repos tests including plan, apply, update, webhook-secret persistence
  • go test ./internal/workflow/ — Disable/Reenable
  • go build ./cmd/mantle && ./mantle repos --help — surfaces the full command set
  • Manual: register a private repo with --webhook-secret, configure the webhook in GitHub, push a commit, verify immediate sync
  • Manual: delete a repo, verify the clone directory is gone and git_repo_workflows rows are cascaded

Closes the GitOps core loop of #16. Plan D tracks remaining polish.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Apr 19, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5615110a-8a81-44cb-8061-3fea898a15b0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/git-sync-complete

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@michaelmcnees michaelmcnees force-pushed the feat/git-sync-engine branch from 36aa741 to 37f6147 Compare May 30, 2026 02:34
michaelmcnees and others added 13 commits May 29, 2026 22:35
Adds disabled_at column to workflow_definitions and git_repo_workflows
join table to support prune behavior for GitOps sync (issue #16).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add Pruned field to Report, call RecordSeen+Reenable per file, and
drive Disable+audit+RemoveSeenRecords for stale workflows when Prune
is set. Use DB clock for syncStart to avoid Go/Postgres time skew.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Registers POST /hooks/git/<repo-id>: HMAC-SHA256 verifies against
webhook_secret, emits git.push.received audit event, fires SyncRepo
in a goroutine for immediate 202 response. Adds Store.GetByID
(no team_id filter) and wires RepoStore/GitDriver into Server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a cloneBasePath parameter to Store.Delete so the CLI removes the
on-disk clone directory (filepath.Join(cloneBasePath, repoID)) after the
DB transaction commits. Filesystem failures are logged via slog.Warn but
never fail the call — an orphan directory is preferable to an orphan row.
The CLI resolves the path from cfg.Storage.Path (same fallback as serve).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds PlanRepo (dry-run: pull + discover + classify without any DB writes
or audit events), and exposes it via `mantle repos plan <name>`. Also adds
`mantle repos apply <name>` as an explicit manual-sync alias for operators
using auto_apply:false repos. Refactors newReposSyncCommand to share
buildSyncDriver so all three commands use identical driver-selection logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add --webhook-secret flag to repos add and repos update (blocker)
- Remove stray fmt.Println from webhook test
- Clarify plan/apply/update help text, mention commands in repos Long
- Sanitize last_sync_error for pull/discover failure paths

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gosec G118 (CWE-400) flags goroutines that use context.Background()
when a request-scoped context is available. Two fixes:

- emitGitAudit: called synchronously while the request is still in
  flight, so r.Context() is correct and preserves trace/deadline info.
- background SyncRepo goroutine: context.WithoutCancel(r.Context())
  propagates request values (e.g. trace IDs) without tying the
  goroutine's lifetime to the HTTP request, so it survives after the
  202 response is sent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@michaelmcnees michaelmcnees force-pushed the feat/git-sync-complete branch from e2972f0 to 58aaec0 Compare May 30, 2026 02:37
- Add WebhookSecret field to Repo struct (was dropped during rebase)
- Restore reposCtx to return (context.Context, *config.Config); Plan C
  tests call store.Get with a real context — the 1-return form was wrong
- Update govulncheck exclusions to include go-git transitive dep vulns
  (matching feat/git-sync-engine) and refactor with KNOWN variable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@michaelmcnees michaelmcnees deleted the branch feat/git-sync-engine May 30, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant